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(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly 
affect or be directly affected by or have a bearing on the decision in the pending appeal 
is contained in the brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments after final rejection 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 

The appellant's statement of the issues in the brief is correct. 

(7) Grouping of Claims 

The rejection of claims 91-130 stands or falls together. 

(8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief . 

(9) Prior Art of Record 
6,504,951 Luoetal. 1-2003 
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(10) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claims 91-130 are rejected under 35 U.S.C. 102(e) as being anticipated by US 
6504851 to Luo et al. 

As to claim 91, Luo discloses a method for modifying an image having pixels, 
comprising: 

automatically identifying a main subject (candidate sky ) of the image (fig. 6, 73- 
74, col. 9 lines 25-35); and 

automatically altering pixel values (eliminating pixel having texture above 
threshold or fills voids of connected components) to emphasize the main subject, said 
altering following identifying (abstract, fig. 6, 75, col. 9 lines 38-39, 57-67 note that 
thresholding, eliminating, and region growing are conducted after identifying the main 
subject, candidate sky) 

As to claim 92, Luo further discloses the step of identifying comprises 
segmenting the image into a plurality of regions (col. 9 lines 25-43). 

As to claim 93, Luo further discloses generating a belief map of said region (fig. 6 
71, col. 9 lines 15-27, col. 8 lines 60-64). 

As to claims 94-97, Luo further discloses altering color saturation (col. 16 lines 2- 
14), hue (color, col. 10 lines 32-67, note that adding blue to pixels in the void or 
eliminating color pixels not in sky (set to respective luminance) can be viewed as 
altering hue), luminance and pixel blur (abstract, col. 10 lines 48-67, col, 15 lines 26- 
37). 

As to claims 98-99, Luo further discloses altering emphasizes the main subject 
by altering values that are part of said main subject (abstract, fig. 6, 75, col. 9 lines 38- 
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39, 57-67, col. 10, lines 52-67 note that filling voids of connected components that is 
main subject, and eliminating connected components that are not blue sky). 

As to claims 100-101 , Luo further discloses desaturating the pixels that are not a 
part of said main subject (col. 16 lines 3-14) and in desaturating further comprises: 
calculating luminance values of the pixels the are not a part of main subject and 
replacing the color values of the pixels that are not a part of the main subject with 
respective luminance values (abstract, fig. 6, 75, col. 9 lines 38-39, 57-67, col. 15 line 
65-col. 16 line 14) 

As to claims 102-104, Luo further discloses enhancing saturation of the main 
object pixels (sky) (col. 3 lines 6-21, col. 16 lines 3-7). 

As to claim 105, Luo further discloses segmenting the image into plurality of 
regions and assigning a believe value to the pixels corresponding to the level of 
saliency (col. 9 lines 15-42). 

As to claims 106-120, the claims are corresponding system claims to claims 91- 
105. The discussions are addressed with regard to claims 91-105. 

As to claim 121, Luo discloses a method of modifying an image having pixels, 
comprising: 

automatically generating one or more belief values, each being associated with 
one of plurality of regions of the image, said belief values each being related to the 
probability that the associated region is a main subject of the image (fig. 2, 201 , fig. 4a- 
d, col. 8 lines 47-64); 

following said generating, automatically altering pixel values in the plurality 
regions of the image, in accordance with the associated belief values (fig. 6, 75, col. 10 
lines 49-67). 

As to claim 126, the limitations are addressed with regard to claim 121. 
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As to claims 122-125 and 127-130, the limitations are addressed with regard to 
claims 94-105. 

(11) Response to Argument 

(A) The following discussion relates to the rejection of claims 91-130 under 35 U.S.C. 
102(e) as being anticipated by US 6504951 to Luo et al. 

1 . Appellants' argument — Appellant argues that "Luo et al. does not disclose 
automatically identifying a main subject of the image" because Luo expressly mentions 
"[t]his information may be used as a guide for manipulation tasks such as orientation of 
the image or excluding the sky region in detecting the main subjects in the image ." (col. 
1 , lines 48-52) (brief page 3, paragraph 3, emphasis by Appellant). 

Examiner's response — The Examiner respectfully disagrees with Appellant's 
assertion regarding Luo. First, the language cited by Appellant is in the background 
section of Luo's patent to show that, in the prior arts, "sky regions can usually be 
excluded", which is problem Luo's patent tried to solve. In fact, Luo expressly and 
clearly discloses detecting sky in the title "Method for detecting sky in images" 
(emphasis by Examiner). In addition, in the section of summary of the invention, Luo 
expressly and clearly states that [t]he invention provides a robust sky detection 
system which is based on color hue classification, texture analysis, and physics- 
motivated sky trace analysis." (col. 2 lines 59-60) (emphasized by Examiner). Moreover, 
Appellant, in the instant case, further expressly defines the identifying comprises 
segmenting the image into regions and generating belief map of the regions (claims 92- 
93). Finally, Luo also expressly and clearly discloses segmenting the image into regions 
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(col. 9 lines 25-43) and generating belief map for the regions (col. 8 lines 60-64). Thus, 
Luo actually anticipates the broadly claimed language "automatically identifying a main 
subject of the image". 

Therefore, the rejection under 35 USC 102(e) is proper. Appellant's argument is 
based on an apparent misinterpretation of Luo, and the result is this appeal brief. 

2. Appellants' argument — Appellant also argues that "the interpretation of the 
disclosure of the reference should be given exceptional weight" because Luo is the first 
named inventor of US 6504851 . (brief page 3, paragraph 2). 

Examiner's response — The Examiner respectfully disagrees with Appellant. 

First, Appellant does not cite any legal authority to support his argument. In 
addition, how much weight is "exceptional weight"?, Examiner could not assess what 
the extent of reliance on Mr. Luo's subjective interpretation should be. Finally, even if 
assuming the Examiner should give exceptional weight to Mr. Luo's interpretation, 
Examiner believe that US 6504851 is clear on its face such as the title "method of 
detecting sky in images", segmenting the image, and generating belief map etc. ant 
contrary interpretation by Mr. Luo at this point is unpersuasive. 

3. Appellants' argument — Appellant still argues that Luo does not disclose 
automatically altering pixel values of said images to emphasize said main subject, said 
altering following said identifying because a) H [s]ince the process is intended to find sky 
regions, it by definition cannot take place after the sky has been identified. The 
eliminating pixels step in Luo et al. is an intermediate step in finding sky regions." 
(page 4, paragraph 3, emphasized by Examiner); and b) the step of eliminating pixels 



Application/Control Number: 09/642,533 Page 7 

Art Unit: 2623 . 

having texture above a threshold or filling voids of connected components are not done 
to alter pixel values for emphasizing the sky regions. Emphasizing a region relative to 
other regions does not mean, in the context of the present application, eliminating the 
other regions; it simply means the other regions are not emphasized." (brief page 4, 
paragraph 4). 

Examiner's response — The Examiner does not agree with Appellant. 
Regarding a), first of all, Examiner would clarify that "find sky regions" is not equivalent 
to "identifying a main subject" of claimed language. Appellant apparently tries to argue 
"find sky region" is equivalent to "identifying a main subject" as claimed and then to rely 
on the definition of "find sky regions" to make his argument. However, Appellant 
arbitrarily exchanges the concepts here. Examiner would emphasize that "find sky 
regions" is not in the claimed language. The word "find" is not equivalent to the word 
"identify" defined in the claims, i.e., segmenting images into regions and generating 
belief map of these regions. In addition, the term "sky region" is not equivalent to the 
term "main subject" that has much broader meaning. Thus, Appellant wrongly argues 
that 'find sky regions" is the same as identifying main subject as claimed. Appellants 
argument further shows a mischaracterization of Luo and misinterpretation the claimed 
language. Moreover, Examiner has construed the main subject as equivalent to Luo's 
"candidate sky region" in the final rejection. Luo first identifies the candidate sky regions 
by segmenting the image into regions and generating a belief map of segmented 
regions, and then using region growing methods to refine the main subject, "[sjmall, 
isolated sky regions are ignored" (col. 10 lines 49-67). The refinement is inherently 
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emphasizing the candidate sky region to find "true sky region" (col. 16, lines 5-7). 
Finally, Examiner's claim construction of "main subject" is correct even in light of the 
specification that mentions "the output of main subject detection algorithm is a list of 
segmented regions ranked in descending order of the likelihood (or belief) that each is a 
main subject." (page 13 lines 13-15, specification). Therefore, the step of Luo's 
refinement is read on the claim language of "automatically altering pixel values of said 
images to emphasis said main subject" and the altering is done after identifying. 

Regarding b), "emphasizing" is a general term. To 'filling in holes and extend 
boundaries" of region growing clearly emphasizes the candidate sky regions that is the 
main subject (col. 10 lines 50-52). In addition, Luo further expressly teaches that "[t]his 
is especially useful where "marginal" pixels may have sky-color belief values that barely 
fail the global threshold but are close enough to the belief values of the neighboring 
pixels that have passed the initial global threshold." (col. 10 lines 52-56). Finally, 
Robinson clearly and expressly teaches the emphasizing is done after identifying to 
refine the marginal pixels, to fill in holes, and extending boundaries, etc. Therefore, 
Appellant's argument is unpersuasive . 

For the above reasons, it is believed that a prima facie case has been fully 
established and the rejections should be sustained. 



Respectfully submitted 
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December 21, 2004 



Appeal Conferees: 




